Decoupling Learning Rules from Representations

In the artificial intelligence field, learning often corresponds to changing the param-eters of a parameterized function. A learning rule is an algorithm or mathematicalexpression that specifies precisely how the parameters should be changed. Whencreating an artificial intelligence system, we must make two decisions: what repre-sentation should be used (i.e., what parameterized function should be used) andwhat learning rule should be used to search through the resulting set of repre-sentable functions. Using most learning rules, these two decisions are coupled in asubtle (and often unintentional) way. That is, using the same learning rule with twodifferent representations that can represent the same sets of functions can result intwo different outcomes. After arguing that this coupling is undesirable, particularlywhen using artificial neural networks, we present a method for partially decouplingthese two decisions for a broad class of learning rules that span unsupervisedlearning, reinforcement learning, and supervised learning.